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METHOD FOR VISUALLY 7^ AUDIBLY REPRESENTING 
COMPUTER INSTRUCTIONS FOR EDITING VIDEO 

Cross-Ref erence to Related Applications 
This application claims the benefit under 35 U.S.C. 
§120 of previously-filed, co-pending application Serial 
No, 07/867,125, filed April 10, 1992 and entitled 
"Method for Providing Visual Information with 
Instructions for Editing a Video Tape or Film Program," 
which is expressly incorporated by reference. 

Field of the Invention 
The invention belongs to the field of video and film 
editing, More particularly, the invention is related to 
methods for enabling a human user to understand computer 
instructions for editing video. 

Background of the Invention 
Completed video and film programs are generally 
composed of segments from several sources. The programs 
are typically assembled by an editor who views the 
sources of material that are available and chooses the 
segments that will make up the final program. However, 
the program is not usually edited at the same place or 
time as the physical production of the final video tape 
or film; instead, the final production occurs at a 
facility equipped to produce the high-quality editing 
that is required for the final product. Therefore, the 
original editor of the program must generate a set of 
editing instructions to be used in the construction of 
the final program which is commonly automated using 
computer technology, 

A set of editing instructions for video or film 
programs is often produced in a format called an edit 
decision list (EDL) . A conventional EDL consists of a 
sequence of editing instructions, each of which is a 
computer instruction for a computerized edit controller 
which assembles a final program from source material. 
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An editing instruction represents an event description, 
where each event is a transition to a new^ program 
segment. There are a number of available EDL formats 
(CMX^ GVGr Sony, etc* ) r but each conveys similar 
information. The event description contains such 
information as the source of the new program segment, 
the time codes describing both the portion of the source 
that will be recorded and its destination in the final 
program,, and the type of edited transition that is to be 
used from the previous segment. Using the information 
represented by editing instructions, the final program 
can be automatically constructed from the several 
sources of program material. 

However, there are several problems with using 
conventional EDL's for video editing. Although EDL 
formats were created in a form which allowed them to be 
read by people, they are not in a form that is readily 
understood, as they are intended primarily to be 
computer instructions. A conventional EDL contains only 
alphanumerical information describing the video or film 
program transitions. This information is usually 
sufficient to produce the program. However^ it is 
impossible from a conventional EDL to visualize the 
final program or its sources. For example, if an EDL 
reguires modification, . the original source material must 
be examined to determine the effects of any 
modification. This problem could arise, for instance, 
if an EDL was created to produce a sixty-minute program, 
but the appropriate program length was subseq[uently 
determined to be fifty-five minutes, thus requiring 
material to be cut. If the original editor (i.e., the 
person who produced the EDL) is \inavailable or does not 
remember the exact nature of the segments of the 
program, it is impossible to determine from the EDL 
alone which portions of the program should be cut. 

Furthermore, because the video or film program 
represented by the EDL cannot be readily visualized, in 
order to view the final program it must be actually 
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constructed from the EDL. For example, if the final 
program is a television commercial, it is common to have 
it approved by the advertiser. Because the program 
represented by the EDL cannot be readily visualized, the 
final video tape or film must be sent to the advertiser 
for prior approval, which is an expensive and 
time-consuming process. 

Summary of the Invention 
The invention disclosed herein is a method for 
visually and audibly representing computer instructions 
for editing video which provides audio and/or visual 
information along with editing instructions for a video 
tape or film program. The method involves, first, 
choosing the segments of video tape or film material to 
be incorporated into a video tape or film program, and 
then storing identifications of the sources of those 
segments of the program. Then, audio and/or visual 
information representative of each segment is stored so 
as to be associated with its corresponding 
identification of the source of the segment. 

In one embodiment of the invention, the 
identification of the source of each segment includes an 
indication of the source tape or film of the segment and 
time codes indicating the points at which the segment 
begins and ends. The segment source identification can 
also indicate the type of transition (wipe, dissolve, 
etc.) that is to be used from the previous segment. 

In a preferred embodiment of the invention, both 
the identifications of the segment sources and the 
visual representations are stored in computer-accessible 
forms, and the segment source identification includes a 
computer-readable pointer to the audio and/or visual 
representations for that segment. The visual 
representation is a sequence of images, and both the 
audio and visual representations can be in a digitized 
form. In a preferred embodiment, the invention provides 
a signal whereby an operator can request playback of the 
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seguence of images and sound for any source segment . 
Ideally, bo1:h the segment source identifications and the 
digitized image and sound sequences are stored in the 
same computer file. • 

One possible means of implementing the storing and 
playback of the image sec[uences for the invention is to 
use QuickTime and WildMagic on the Apple Macintosh 
computer. QuickTime provides a means of playing back 
the sequence of images v and WildMagic 'allows the image 
sequence to be stored in a text file. 

Brief Description of the Drawing 
In the drawing. 

Figure 1 is a block diagram of a computer system 
suitable for use with the present invention; 

Figure 2 is a conventional edit decision list; 
. Figure 3 is a representation of a visual edit 
decision list with a digitized image sequence (shown in 
the figure as a still picture) associated with each 
element ; and 

Figure 4 is a graphical representation of a timeline 
representing a visual edit decision list. 

Detailed Description 

A conventional edit decision list (EDL) is a series 
of computer instructions for an edit controller 20 (Fig. 
1) which produces an edited video or film program from 
several sources. The EDL is a list of editing 
instructions, each of which represents one segment of 
the program. Each editing instruction indicates the. 
source of that segment (e.g., by the name of the video 
tape or film on which it is recorded) and the portion of 
the source, indicated by start and stop times, that 
makes up the segment. Edit decision lists are commonly 
generated using a computerized editing system 15 such as 
shown in Fig, 1\ 

such a system 15 commonly includes a computer 22 
with a volatile, typically random— access , memory 24 
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coruiected to a central processing unit (CPU) 26 via a 
bus 28. Conventional computer systems, as well as those 
specially designed for video editing, may be used as the 
system 15 for the purpose of editing. More conventional 
computer systems may also include a printer 32 and 
non-volatile memory or storage 30, such as a hard disk 
or optical disk. The computer system 15 may be 
connected to an edit controller 20 to provide it with an 
EDL from which a video program may be -created. The 
computer system 15 may also include a video display 34. 

Figure 2 shows a typical EDL as it could appear on a 
computer display or as printed. This EDL is in CMX 340 
format. (It should be noted that the particular EDL 
format used is not important for the invention, which is 
applicable to edit instructions generally.) This 
particular EDL is composed of eight editing 
instructions, each labeled by reference numeral i, in 
this particular EDL, track type labels 2 indicate that 
one of the instructions describes the audio (A) portion 
of the program and the other seven instructions describe 
the segments of the video (V) portion. Each 
instruction, whether audio or video, indicates the 
origin of that segment by source label 3 and by source 
start time 4 and source stop time 5 of the segment. 
Similarly, destination start time 6 and destination stop 
time 7 indicate where that segment is to be recorded on 
the final program. Finally, transition type label 8 
describes the type of transition (wipe, dissolve, etc) 
that is to be used in moving to the segment described in 
the current editing instruction from the previous 
program segment. 

To take a particular example of the EDL format, 
consider Element 3. Since track type label 2 for this 
element is "V" , this element is a video segment. It is 
found on the tape or film labelled 1100511 from time 
code 06:46:25:03 to time code 06:46:26:03. Therefore, 
it is one minute long, and it is to be recorded from 
time code 01:00:01:00 to time code 01:00:02:00 on the 
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"RECORD" tape or film. The edit transition from the 
previous segment to this one on. the final tape is to be 
a cut, indicated by a "C" in transition-type label 8. 

In the invention, the conventional EDL format is 
retained, but visual information is added. In the 
preferred embodiment of the. invention, digital image 
sequences are added to the conventional EDL in a 
computer data file. Figure 3 shows the same EDL as in 
Figure 2 with the visual information added. Again, this 
is how the EDL would appear on a computer screen or 
display 34 or as printed via printer 20, though it 
should be noted that the images in the figure are very 
coarse and images on a computer screen are typically 
clearer. The visual BDL contains, as can be seen in 
Figure 3-, all of the alphanumeric a 1 information in the 
conventional EDL, with the addition of a comment field 
9, which can be used as shown in the figure to label the 
segment. Most importajitly, though^ it can also be seen 
in Fig. 3 that for each editing instruction in the EDL 
there is an image 10, In Fig. 3, where only a still 
image can be displayed, each element is shown with the 
starting image of that segment (in elements 1 and 2, the 
image is black because the program begins by dissolving 
from black) . 

A sequence of images and sound that can be played 
back at the operator's request is preferably provided. 
This is possible in the embodiment of the invention in 
which the EDL is stored and read on a computer. 
Digitized image and sound sequences can be stored in 
computer files from which they can be played over the 
computer display 34 . (Fig. 1) and audio speakers. In the 
preferred embodiment of the invention, each instruction 
in the EDL points to digitized sequences of video images 
and sound for that s.egment. The operator reading the 
EDL can then play back the visual and audio portions of 
any segment. (Again, this capability is available only 
when the operator is using the EDL on the computer. 
When the EDL is printed to paper, only a still image is 
available. ) 
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The visual and audio sequences may be implemented in 
any of several ways. In general, the only requirements 
are for the ability to store the audio and visual 
sequences and to be able to reference those sequences 
from a file that contains the EDL text. A variety of 
general- and special-purpose computers can provide these 
capabilities. For example. Avid Technology, Inc., 
Burlington, Massachusetts, has a software-based digital 
picture and soxand compression system that can be used to 
generate and store such sequences. Apple Computer Co., 
Cupertino, California, also markets two products that 
can be used for this purpose, QuickTime and WildMagic 
for Apple Macintosh computers. QuickTime provides the 
Macintosh system with the capability to play a sequence 
of images stored in the computer memory on the display, 
and WildMagic allows a movie file, or, more precisely, a 
pointer to a movie file, to be placed in a Macintosh 
text file. (Use of these products is well-known in the 
art.) Hence, using these two products, a digital movie 
file (created, for example, while editing the material 
and create the EDL) can be placed in the EDL text file. 
Normally, although the text file will contain a 
reference to the entire movie, only a single image will 
be visible on the screen. However, using the capability 
of QuickTime, the operator can click on that image 
(i.e., use the Macintosh mouse to point to the image, 
and then press the mouse button to indicate that 
location to the Macintosh) to play the movie while 
remaining in the EDL computer to see the digitized movie 
of any segment of the EDL. 

In another embodiment of the invention, the visual 
EDL document has three main sections: Header, Timeline, 
and Events List. 

The Header section contains information which 
describes the program as a whole. It may contain the 
program's title, the name of the application which 
created the list, format version number, and creation 
date. In addition to standard information always 



wo 93/21635 



PCT/US93/03360 



-8- 

present in the header section, the user may add 
supplemental information such as a short summary. 
Various kinds of formatting and control data may also be 
present, such as the text font to be used or point 
size. The following is one example of a possible header 
section: 

Avid Visual EDL Format 

Inc. Generated by an Avid Media Composer, Version 5.0 
VEDL Format : 1 . 0 
Title ; A Tahiti Travelog 
. Creation Date: February 20, 1992 

The timeline section is a graphical depiction of 
the video program as a whole, such as shown in Fig. 
4. Records tracks 40 (Video, Audio, Graphics, etc.) 
are arranged vertically, and program time is laid out 
horizontally. Time is represented by 
hour -minute— second-frame timecodes 42. Edits and 
transition effects are represented graphically with 
the use of icons 44 and other symbols. One possible 
representation, as a timeline, of part of the program, 
created by the EDL of Figs. 2 or 3, is shown in Fig. 
4, The generation of a timeline can be done using 
common computer graphic programming techniques. 

Following the timeline is the edit decision 
list. In a conventional EDL, each event would be 
represented by one or two lines of specially formatted 
number and letter sequences which indicate the 
channels present, source identifier, the event type, 
source and record timecodes. In this embodiment, the 
conventional EDL format is retained and augmented with 
a digital movie clip which illustrates each event with 
moving pictures and sound.. Four typical events are 
shown here. 

Another aspect of the invention is a "Viewer" 
application for viewing a visual EDL in accordance 
with the invention. The Viewer allows EDL documents 
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to be displayed on a color or monochrome computer 
screen. Paging, scrolling, goto events, and other 
functions which may be provided using common 
techniqpjes allow various sections of the document to 
be displayed. Event clips can be activated and played 
to illustrate the event's content. A system capable 
of performing such functions is described above. In 
addition to displaying visual EDL documents, the 
Viewer application may also: extract -information to 
file using conventional EDL formats; print using 
conventional EDL formats; print a graphical 
representation; transmit EDL information; and receive 
EDL information 

Extraction of conventional EDL data and its 
presentation in traditional file and print forms 
allows the visual EDL document continues to serve as 
an interchange format to other editing systems. Given 
a standard format for a visual EDL, common computer 
programming techniques can be used to generate a 
program to extract the conventional EDL information. 
This portability is highly useful, since the primary 
purpose of any EDL is to allow a video program to be. 
assembled by an edit controller. 

The visual EDL format is a completely new EDL 
format. As such, it is not currently supported by any 
other vendor of editing equipment, nor is it likely to 
be in the near future. A much closer level of 
cooperation among vendors is needed to handle video 
material than is need for text or sound. For this 
reason, by providing a function to extract EDL 
information and re-format it into one of several 
traditional EDL formats (CMX, GVG, Sony, etc.), the 
visual EDL document continues to serve as an 
interchange document. Extracted EDL files are 
machine-ready for loading into a target on-line system 
for creating a final video program. 

The complement to extracting a traditional EDL 
file is printing in a traditional format. Printed 
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files are quite useful during an on-line session for 
making notes, checking off edits assembled, etc. Most 
on-line editors are used to dealing with EDL 
information in one of these formats. 

If a computer riinning the visual EDL Viewer 
application is physically near the on-line system 
(such as on a portable computer on the console, for 
example), the ability to format an EDL for the target 
system and transmit it serially to the system could be 
provided using well-known techniques. This capability 
is already provided by several EDL utilities and 
greatly enhances the effective use of VEDL documents. 

In addition to a traditional EDL printout, the 
visual EDL document may be printed in a graphical form 
as well. This printout would include the timeline 
diagram, head frames of each video clip and any other 
visual information contained in the document. 
Naturally, sound and motion information cannot yet be 
reproduced on paper. Printing of such visual EDLs can 
be performed using the computer system described above. 

Finally, the Viewer may allow new EDL data to be 
received from an on-line system and stored in a VEDL 
document. Blank clips are included as movie place 
holders for each edit instruction. A graphical 
timeline diagram is- created and included using common 
graphic techniques and common processing steps, as 
found in conventional edit controllers to construct a 
timeline. This allows capture of EDL information from 
the on-line system so it can be transported back to a 
system capable of adding video clips, etc. 

There are several uses for visually represented 
EDLs. First, if modifications must be made to the EDL 
to produce the final program, the invention simplifies 
the process by allowing an editor to visualize the 
program from the edit decision list. For example, if 
a final program is intended for a four-minute 
television news spot, and its EDt. provides 
instructions for a four-minute program, and, a late 
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story roust be added to the news, it may be necessary 
to reduce the length of the original program. The 
person who had originally created the EDL for that 
program might remember which portions of which 
segments should be cut. But if that person is not 
available or does not remember each segment exactly, 
the invention allows another person to quickly view 
each segment to determine which portions of each 
segment are most easily omitted. With- conventional 
EDLs, such a change would regiiire using the EDL 
instructions to actually run the source material prior 
to deciding what to cut, a much more time-consuming 
process . 

Another example of the usefulness of the 
invention is in the creation of commercials. When an 
agency creates a proposed television commercial, it 
must generally be approved by the advertiser. The 
advertiser could not use a conventional EDL to view 
the commercial, so it would be necessary to produce a 
final version for approval. The visual EDL invention, 
in contrast, allows the advertiser to see the final 
result, albeit in a somewhat less high-quality format, 
without the expense of producing a final video program 
that may not be approved. Moreover, changes could be 
suggested from viewing the EDL, which would greatly 
improve the turnaroxind time between successive 
versions of the commercial. Finally, only when the 
visual EDL is approved will it be necessary to have 
the final video program produced. 

Having now described a few embodiments of the 
invention, it should be apparent to those skilled in 
the art that the foregoing is merely illustrative and 
not limiting, having been presented by way of example 
only. Numerous modifications and other embodiments 
are within the scope of one of ordinary skill in the 
art and are contemplated as falling within the scope 
of the invention as defined by the appended claims. 

What is claimed is: 
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Claims 

1. A method for providing audio and/or visual 
information with instructions for editing a video 
program, comprising: 

a. choosing segments of video material to be 
incorporated into a video program; 

b. storing an identification of the source of 
each segment of the program; and 

c. storing with the identification of the 
source of each segment an audio and/or visual 
representation of that segment. 

2. The method defined by claim 1, wherein the 
identification of the source of a segment includes an 
indication of the source tape or film for the segment 
and time codes indicating the points at which the 
segment begins and ends . 

3. The method defined by claim 2, wherein the 
identification of the source of a segment includes an 
indication of the type of transition (wipe, dissolve, 
etc.) to be used from the previous segment. 

4. A method for providing audio and/or visual 
information with instructions for editing a video tape 
or film program^, comprising: 

a. choosing segments of video tape or film 
material to be incorporated into a video tape or film 
program; 

b. storing an identification of the source of 
each segment of the program in a computer-accessible 
form; 

c. storing an audio and/or visual 
representation of each segment in a 

computer— accessible form; and d. storing with the 
identification of each segment source a 
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computer-readable direction to the audio and/or visual 
portion of the representation of that segment. 

5. The method defined by claim 4, wherein the 
visual representation for a segment is a sequence of 
images , 

6. The method defined by claim 5, wherein the 
audio and/or visual representations for a segment are 
in a digitized form. 

7. The method defined by claim 6, wherein the 
segment source identifications and the digitized audio 
and/or visual representations are stored in a single 
computer file. 

8. The method defined by claim 6, further 
comprising monitoring a signal whereby an operator can 
request playback of the audio and/or visual 
representations of for any segment. 

9. A method for visually representing computer 
instructions for editing a video program, comprising: 

a. choosing segments of video material to be 
incorporated into a video program; 

b. storing an identification of the source of 
each segment of the program; and 

c. storing with the identification of the 
source of each segment an audio and/or visual 
representation of that segment. 

10. The method defined by claim 9, wherein the 
identification of the source of a segment includes an 
indication of the source tape or film for the segment 
and time codes indicating the points at which the 
segment begins and ends. 
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11. The method defined by claim 10, wherein the 
identification of the source of a segment includes an 
indication of the type of transition (wipe, dissolve, 
etc.) to be used from the previous segment. 

12. A method for visually representing computer 
instructions for editing a video program, comprising: 

a. choosing segments of video tape or film 
material to be incorporated into a video tape or film 
program; 

b. storing an identification of the source of 
each segment of the program in a computer-accessible 
form; 

Cr storing an audio and/or visual 
representation of each segment in a 
computer-accessible form; and d. storing with the 
identification of each segment source a 
computer-readable direction to the audio and/or visual 
portion of the representation of that segment. 

13. The method defined by claim 12, wherein the 
visual representation for a segment is a sequence of 
images . 

14. The method defined by claim 13, wherein the 
audio and/or visual representations for a segment are 
in a digitized form. 

15. The method defined by claim 14, wherein the 
segment source identifications and the digitized audio 
and/or visual representations are stored in a single 
computer file. 

16- The method defined by claim 14, further 
comprising monitoring a signal whereby an operator can 
regiiest playback of the audio and/or visual 
representations of for any segment. 
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17. A method for visually representing computer 
editing instructions of a video program, comprising 
the steps of: 

a. generating and storing in a memory, a 
sequence of computer editing instructions, each 
identifying a source video and a segment of the source 
video; 

b. storing, with each computer editing 
instruction a visual representation of the segment of 
the computer editing instruction. 

18. The method of claim 17 further comprising 
the step of generating and storing, with the plurality 
of editing instructions, a graphical timeline 
representative of the video program. 

19. A system for manipulating visual 
representations of computer instructions for editing 
video and audio, the system comprising: 

a. means for receiving a visual edit 
decision list including edit instructions and 
associated video information; 

b. means for displayng the visual edit 
instructions, including means for displaying the video 
information. 

20. The system of claim 19 further comprising 
means for extracting the edit instructions from the 
visual representation of the computer instructions. 
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